Systems and methods for improved operations of ski lifts

ABSTRACT

Systems and methods for improved operations of ski lifts increase skier safety at on-boarding and off-boarding locations by providing an always-on, always-alert system that “watches” these locations, identifies developing problem situations, and initiates mitigation actions. One or more video cameras feed live video to a video processing module. The video processing module feeds resulting sequences of images to an artificial intelligence (AI) engine. The AI engine makes an inference regarding existence of a potential problem situation based on the sequence of images. This inference is fed to an inference processing module, which determines if the inference processing module should send an alert or interact with the lift motor controller to slow or stop the lift.

TECHNICAL FIELD

The technical field relates to ski lifts, and particularly to improvingoperations of ski lifts.

BRIEF SUMMARY

Systems and methods for improved operations of ski lifts increase skiersafety at on-boarding and off-boarding locations by providing analways-on, always-alert system that “watches” these locations,identifies developing problem situations, and initiates mitigationactions. One or more video cameras feed live video to a video processingmodule. The video processing module feeds resulting sequences of imagesto an artificial intelligence (AI) engine. The AI engine makes aninference regarding existence of a potential problem situation based onthe sequence of images. This inference is fed to an inference processingmodule, which determines if the inference processing module should sendan alert or interact with the lift motor controller to slow or stop thelift.

A computerized method for improved ski lift operations may be summarizedas including capturing, by at least one computer processor, digitalvideo of one or more of on-boarding and off-boarding operations of a skilift; generating, by at least one computer processor, as the ski lift isoperating, a plurality of digital images of the one or more ofon-boarding and off-boarding operations of the ski lift based on thecaptured digital video; automatically detecting, by at least onecomputer processor, in real-time as the digital video is being captured,as the ski lift is operating, a potential problem situation in one ormore of an on-boarding area and an off-boarding area of the ski liftbased on the plurality of digital images; and initiating an action, bythe at least one computer processor, as the ski lift is operating, toaddress the potential problem situation in the one or more of theon-boarding area and off-boarding area of the ski lift while thepotential problem situation still exists.

The method may further include analyzing, by at least one computerprocessor, the plurality of digital images as the ski lift is operating,wherein the automatically detecting the potential problem situation isbased on the analysis of the plurality of digital images. The analyzingthe plurality of digital images as the ski lift is operating, mayinclude recognizing in the plurality of digital images a sequence ofevents that represents an abnormal on-boarding or off-boarding processat a scene of the ski lift that includes the one or more of theon-boarding area and an off-boarding area of the ski lift. The sequenceof events that represents an abnormal on-boarding or off-boardingprocess may include one or more of a lift rider falling whileon-boarding the ski lift; a lift rider falling while off-boarding theski lift; a lift rider lying or sitting the ground in a lift chairloading zone; a lift rider being dragged by a ski lift chair; a ski poleor other equipment of a lift rider being caught in a ski lift chair; alift rider proceeding to on-board the ski lift late; a lift riderfailing to exit off-boarding area; a lift rider being in an abnormalposition on a lift chair after on-boarding the ski lift; a lift riderstarting to slip off a lift chair after on-boarding the ski lift; a liftrider leaving a lift chair loading zone as a lift chair is approachingthe lift rider; a lift rider under a pre-determined size in a lift chairloading zone as a lift chair is approaching the lift rider; a number oflift riders being in a lift chair loading zone that is over apre-determined limit of lift riders for a lift chair; a lift riderfacing a wrong direction in a lift chair loading zone as the lift chairis approaching the lift rider; a lift rider being in an incorrectposition within a lift chair loading zone as the lift chair isapproaching the lift rider; an incorrect number of lift ridersoff-boarding a lift chair compared to a number of lift riders thaton-boarded the lift chair; an incorrect number of lift riders on a liftchair compared to a number of lift riders that were in a lift chairloading zone for the lift chair when the lift chair arrived foron-boarding; a mechanical problem with a lift chair of the ski lift; amechanical problem with operation of the ski lift; and a problem with astructural element of the ski lift. The analyzing the plurality ofdigital images as the ski lift is operating may include analyzing theplurality of digital images using an artificial intelligence enginetrained to assess sequences of digital images and identify normal andpotential problem situations taking place in a scene of the ski liftthat includes the one or more of the on-boarding area and anoff-boarding area of the ski lift.

The method may further include training the artificial intelligenceengine, using sequences of training images of one or more of on-boardingand off-boarding operations of one or more ski lifts, to assess thesequences of digital images and identify normal and potential problemsituations taking place in the scene. Initiating the action to addressthe potential problem situation in the one or more of the on-boardingand off-boarding area of the ski lift while the potential problemsituation still exists may include comparing the detected potentialproblem situation to a plurality of potential problem situations todetermine a match of the detected potential problem situation to amatching one of the plurality of potential problem situations; selectingan action to initiate according to a rule associated with the matchingone of the plurality of potential problem situations; and initiating theselected action to address the potential problem situation in the one ormore of the on-boarding and off-boarding area of the ski lift while thepotential problem situation still exists. The initiating the action toaddress the potential problem situation in the one or more of theon-boarding and off-boarding area of the ski lift while the potentialproblem situation still exists may include initiating electronic sendingof a signal to a motor controller of the ski lift to slow down or stopoperation of the ski lift to address the potential problem situation inthe one or more of the on-boarding and off-boarding area of the skilift. The initiating the action to address the potential problemsituation in the one or more of the on-boarding and off-boarding area ofthe ski lift while the potential problem situation still exists mayinclude initiating of electronic sending of an alert for a ski liftattendant. The sending the alert for the ski lift attendant may includeone or more of sending an audio alert, sending a message-based alert andsending a light-based alert.

A system for improved operations of ski lifts may be summarized asincluding at least one memory; and at least one processor coupled to theat least one memory, wherein the at least one memory hascomputer-executable instructions stored thereon that, when executed,cause the at least one processor to capture digital video of one or moreof on-boarding and off-boarding operations of a ski lift; generate, asthe ski lift is operating, a plurality of digital images of the one ormore of on-boarding and off-boarding operations of the ski lift based onthe captured digital video; automatically detect, as the ski lift isoperating, a potential problem situation in one or more of theon-boarding and off-boarding area of the ski lift based on the pluralityof digital images; and initiate an action, as the ski lift is operating,to address the potential problem situation in the one or more of theon-boarding and off-boarding area of the ski lift while the potentialproblem situation still exists.

The computer-executable instructions, when executed, may further causethe at least one processor to analyze the plurality of digital images asthe ski lift is operating, wherein the automatic detection of thepotential problem situation is based on the analysis of the plurality ofdigital images. The computer-executable instructions, when executed, mayfurther cause the at least one processor to recognize in the pluralityof digital images a sequence of events that represents an abnormalon-boarding or off-boarding process at a scene of the ski lift thatincludes the one or more of the on-boarding area and an off-boardingarea of the ski lift. The analyzing the plurality of digital images asthe ski lift is operating may include analyzing the plurality of digitalimages using an artificial intelligence engine trained to assesssequences of digital images and identify normal and potential problemsituations taking place in a scene of the ski lift that includes the oneor more of the on-boarding area and an off-boarding area of the skilift.

A ski lift controller may be summarized as including a ski lift motorcontroller of a ski lift; at least one memory; at least one videocamera; and at least one processor coupled to the at least one memory,the at least one camera and the ski lift motor controller, wherein theat least one memory has computer-executable instructions stored thereonthat, when executed, cause the at least one processor to capture, viathe at least one video camera, digital video of one or more ofon-boarding and off-boarding operations of the ski lift; generate, asthe ski lift is operating, a plurality of digital images of the one ormore of on-boarding and off-boarding operations of the ski lift based onthe captured digital video; automatically detect, as the ski lift isoperating, a potential problem situation in one or more of theon-boarding and off-boarding area of the ski lift based on the pluralityof digital images; and initiate an action, as the ski lift is operating,to address the potential problem situation in the one or more of theon-boarding and off-boarding area of the ski lift while the potentialproblem situation still exists.

The computer-executable instructions, when executed, may further causethe at least one processor to analyze the plurality of digital images asthe ski lift is operating, wherein the automatic detection of thepotential problem situation is based on the analysis of the plurality ofdigital images. The computer-executable instructions, when executed, mayfurther cause the at least one processor to recognize in the pluralityof digital images a sequence of events that represents an abnormalon-boarding or off-boarding process at a scene of the ski lift thatincludes the one or more of the on-boarding area and an off-boardingarea of the ski lift. The analysis of the plurality of digital images asthe ski lift is operating may include analysis of the plurality ofdigital images using an artificial intelligence engine trained to assesssequences of digital images and identify normal and potential problemsituations taking place in a scene of the ski lift that includes the oneor more of the on-boarding area and an off-boarding area of the skilift.

A non-transitory computer-readable storage medium, having computerexecutable instructions stored thereon that, when executed by the atleast one processor, may be summarized as causing the at least oneprocessor to automatically detect, as a ski lift is operating, apotential problem situation in one or more of an on-boarding andoff-boarding area of the ski lift based on a plurality of digital imagesfrom captured digital video of operation of the ski lift; and initiatean action, as the ski lift is operating, to address the potentialproblem situation in the one or more of the on-boarding and off-boardingarea of the ski lift while the potential problem situation still exists.

The computer executable instructions, when executed, may further causethe at least one processor to capture digital video of the one or moreof on-boarding and off-boarding operations of a ski lift; and generate,as the ski lift is operating, the plurality of digital images of the oneor more of on-boarding and off-boarding operations of the ski lift basedon the captured digital video. The computer executable instructions,when executed, may further cause the at least one processor to analyzethe plurality of digital images as the ski lift is operating, whereinthe automatic detection of the potential problem situation is based onthe analysis of the plurality of digital images.

BRIEF DESCRIPTION OF THE DRAWINGS

The components in the drawings are not necessarily to scale relative toeach other. Like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is an overview block diagram illustrating an exampletechnological environment in which embodiments of systems and methodsfor improved operations of ski lifts may be implemented, according toone example embodiment.

FIG. 2 is a block diagram illustrating elements of an example ski liftproblem detection system, according to one example embodiment.

FIG. 3 is a diagram of an example ski lift at an off-boarding areashowing a ski lift operator in a first position.

FIG. 4 is a diagram of an example ski lift at an off-boarding areashowing a ski lift operator in a second position.

FIG. 5 is a block diagram illustrating a flow of operations betweenvarious elements of an example system for improved operations of skilifts, according to one example embodiment.

FIG. 6 is a flow diagram of a computerized method for improved ski liftoperations, according to one example embodiment.

FIG. 7 is a flow diagram of a computerized method for initiating anaction to address a potential problem situation in one or more of anon-boarding area and off-boarding area of a ski lift useful in improvedski lift operations, according to one example embodiment.

FIG. 8 is a flow diagram of a computerized method for improved ski liftoperations, according to another example embodiment.

DETAILED DESCRIPTION

At a ski area, ski lift on-boarding and off-boarding locations arehigher-risk areas due to the concentration of skiers and the mechanicalnature of the lift. The types of problems that can occur in theselocations include, but are not limited to: falls, which can result ininjury to the falling skier as well as create an obstacle for other thenext set of skiers on- or off-boarding the lift; people failing to getoff of the lift where it is safe to do so and being carried beyond thesafe off-boarding zone; people being dragged by the lift; people notclearing the off-boarding areas and consequently being struck by amoving lift chair; and people failing to successfully load, resulting ina scenario where they try to hang on to the lift, resulting inincreasing danger as the chair rises higher and higher above the ground.

Most, if not all, ski lifts have one or more humans attending these onboarding and off-boarding areas and generally these attendants willrespond to such problems described above, typically by slowing orstopping the ski lift. However, the attendants are not always alert oraware of developing situations, and they can be out of position to takeaction quickly if necessary.

Thus, the systems and methods disclosed herein improve operations of skilifts by increasing skier safety at on-boarding and off-boardinglocations by providing an always-on, always-alert system that “watches”these locations, identifies developing problem situations, and initiatesmitigation actions when necessary. The systems and methods disclosedherein may serve either as a backup or primary monitoring system,capable of taking action if needed, reducing the dependence on humanobservation and freeing the humans to help skiers get back on their feetand out of the way.

FIG. 1 is an overview block diagram illustrating an exampletechnological environment 102 in which embodiments of systems andmethods for improved operations of ski lifts may be implemented,according to one example embodiment. Before providing additional detailsregarding the operation and constitution of systems and methods forimproved operations of ski lifts, the example technological environment102, within which such systems and methods may operate, will briefly bedescribed.

In the technological environment 102, in various example embodiments,the ski lift problem detection system 104 interconnects to at least onevideo camera 102, at least one ski lift motor controller 106 and atleast one ski lift alert system 110. The ski lift problem detectionsystem 104 may also interconnect with one or more mobile devices (e.g.,smartphones, tablet devices, other computing devices, etc.) or remotedevices or systems. The ski lift problem detection system 104 may sendand receive data regarding the monitoring of the ski lift, including,but not limited to, video, images, audio, notification and alert data,analytics, results of image analyses, user interfaces and related data,measurements, detected states the ski lift and lift riders are in,artificial intelligence and machine learning models, sets of trainingdata and verification data regarding machine learning. For example, thevarious data may be delivered using the Internet protocol (IP) suiteover a packet-switched network such as the Internet or otherpacket-switched network or a peer-to-peer network, such as a Bluetooth®network. The underlying connection carrying such data may be or includewireless system operating according to the IEEE 802.11x standards (Wi-Fisystem), wireless local area networking (WLAN) system, cable head-end,satellite antenna, telephone company switch, cellular telephone system,short range radio channel, Ethernet portal, off-air antenna, or thelike. The ski lift problem detection system 104 may receive a pluralityof video or image data from the video camera 102, and may also receivedata from other sources, mobile devices and/or remote servers. This datamay be received or formatted by the ski lift problem detection system104 in various different formats.

Accordingly, the ski lift problem detection system 104 may be a deviceor electronic equipment that is operable to receive and process images,video and/or audio of the ski lift operations and process andcommunicate data regarding the images, video and/or audio as describedherein. Further, the ski lift problem detection system 104 may itselfinclude user interface devices, such as one or more displays and virtualor physical buttons or switches.

Data may be communicated between the ski lift problem detection system104, the video camera 102, the ski lift alert system 110 and the skilift motor controller through suitable communication media, generallyillustrated as communication system 108 for convenience. Communicationsystem 108 may include many different types of communication mediaincluding those utilized by various different physical and logicalchannels of communication, now known or later developed. Non-limitingmedia and communication channel examples include one or more, or anyoperable combination of, Wi-Fi systems, WLAN systems, short rangewireless (e.g., Bluetooth®) systems, peer-to-peer network systems,hardwired systems, communication busses, computer network cabling, widearea network (WAN) systems, the Internet, cable systems, telephonesystems, fiber optic systems, microwave systems, asynchronous transfermode (“ATM”) systems, frame relay systems, digital subscriber line(“DSL”) systems, radio frequency (“RF”) systems, cellular systems, andsatellite systems.

The above description of the technological environment 102 and thevarious devices therein, is intended as a broad, non-limiting overviewof an example environment in which various embodiments of improvedoperations of ski lifts may be implemented. FIG. 1 illustrates just oneexample of a technological environment 102 and the various embodimentsdiscussed herein are not limited to such environments. In particular,technological environment 102 and the various devices therein, maycontain other devices, systems and/or media not specifically describedherein. For example, the techniques and devices described herein may beapplicable in other to detect and address problem situations in otherenvironments, such as, for example, in amusement rides, mass transit,road traffic, and other on-boarding and off-boarding environments.

Example embodiments described herein provide applications, tools, datastructures and other support to implement improved operations of skilifts. Other embodiments of the described techniques may be used forother purposes, including transmitting data to various distributionequipment, computers, peripherals, mobile devices, and other electronicdevices, etc., for further processing and analysis. In the followingdescription, numerous specific details are set forth, such as dataformats, program sequences, processes, and the like, in order to providea thorough understanding of the described techniques. The embodimentsdescribed also can be practiced without some of the specific detailsdescribed herein, or with other specific details, such as changes withrespect to the ordering of the code flow, different code flows, and thelike. Thus, the scope of the techniques and/or functions described arenot limited by the particular order, selection, or decomposition ofsteps described with reference to any particular module, component, orroutine.

FIG. 2 is a block diagram illustrating elements of an example ski liftproblem detection system 104, according to one example embodiment.

Shown in FIG. 2 are one or more video cameras 102 trained on theobservation area (i.e., the on-boarding or off-boarding areas of a givenlift). A computer network of communication system 108 may connect theone or more video cameras 102, one or more processing units (e.g., CPU203) of the ski lift problem detection system 104, the ski lift alertsystem 110 and the ski lift motor controller 106. In some embodiments,the one or more video cameras 102 may be part of or otherwise integratedwith the ski lift problem detection system 104 as one device.

Power is also provided to the one or more video cameras 102, ski liftproblem detection system 104, the ski lift alert system 110 and the skilift motor controller 106 via a powerline connection, power outlet,generator, or one or more batteries, or a combination thereof.Electrical connectivity is provided to the ski lift motor controller 106and may be controlled by the ski lift problem detection system operationmanager 222 via communication system 108 so the ski lift problemdetection system 104 can interact with the lift to slow it down, stopand/or restart it according to output from the inference processingmodule 228. The inference processing module 228 may also determine whento start or speed up the lift based on the inferences that a problemsituation has been resolved or no longer exists. In some embodiments,such control of the ski lift motor controller 106 may have a manualoverride or confirmation step or process optionally provided for the skilift operator to perform before or during starting or speeding up of theski lift.

The one or more video cameras 102 feed live video to the videoprocessing module 224. The video processing module 224 performs thevideo processing described herein and feeds the resulting sequences ofimages to the AI engine 226 (which in some embodiments may be situatedon the same processing unit, such as CPU 202). The AI engine 226 makesan inference regarding the activity represented by the sequence ofimages. This inference is fed to the inference processing module 228,which determines if the inference processing module 228 should send analert or interact with the lift motor controller 106. If so, theinference processing module 228 will interact accordingly with the motorcontroller 106.

In various embodiments, the video processing module 224, inferenceprocessing module 228, and AI engine 226 may be located remotely fromthe ski lift problem detection system 104, such as on one or moreservers in communication with the ski lift problem detection system 104over communication system 108. Maintenance of the video processingmodule 224, inference processing module 228, and AI engine 226 of theski lift problem detection operation manager 222 may follow softwaredelivery practices that enable the AI engine 226, video processingmodule 224, inference processing module 228 to receive software upgradesand adjustments.

An example embodiment of the system for improved ski lift operations“watches” the on-boarding and off-boarding locations of a given ski lift(the “scene”). The inference processing module 228 of the ski liftproblem detection system 104 infers developing situations in the sceneand initiates one or more actions to mitigate or otherwise addresspotential problems when the artificial intelligence (AI) enginedetermines that the situation warrants it. For example, the inferenceprocessing module 228 may compare the detected potential problemsituation to a plurality of potential problem situations to determine amatch of the detected potential problem situation to a matching one ofthe plurality of potential problem situations. The inference processingmodule 228 may then select an action to initiate or perform according toa rule associated with the matching one of the plurality of potentialproblem situations. Such rules may be stored in and accessible from theproblem detection rules and settings storage 216, which, in someembodiments, may be located remotely from the ski lift problem detectionsystem 104 and accessible via communication system 108. The inferenceprocessing module 228 may then initiate the selected action to addressthe potential problem situation in the one or more of the on-boardingand off-boarding area of the ski lift while the potential problemsituation still exists.

The ski lift problem detection operation manager 222 performs videocollection via the video camera 102, performs video processing via thevideo processing module 224, identifies normal or developing problemsituations in the video feed via the artificial intelligence engine 226,and performs motor-control/alert-system integration via the operablyconnected ski lift motor controller 106 and the operably connected skilift alert system 110. Example functions of the ski lift problemdetection system include, but are not limited to: real-time videocapture of skiers on-boarding and off-boarding ski lifts via the videocamera 102; processing of the video into sequences of individual videoframes via the video processing module 224; analysis of the framesequences by the AI engine 226 using an AI engine 226 trained to assesssequences of frames and identify normal or developing problem situationstaking place in the scene; and in the event of developing problemsituations, interaction with the ski-lift motor controller 106 and/orthe ski lift alert system 110 to alert the human lift attendant and/ordirectly initiate a slow-down or complete stop of the ski-lift as thesituation demands.

The ski lift problem detection system 104 performs collection of videofootage from one or more video cameras 102 situated such that theyrecord activity in the area of interest (e.g., on-boarding and/oroff-boarding areas of a given ski lift). In various embodiments,multiple cameras may be used to capture video from various differentlocations and angles of view of the ski lift, including, in someembodiments, areas of the ski lift outside the on-boarding and/oroff-boarding areas.

The video may be collected real-time by the ski lift problem detectionsystem 104 and fed to the video processing module 224, where it isdecomposed into individual frames. The individual frames are cropped andresized by the video processing module 224 and are then assembled inmanageable sequences of frames to be fed to the AI engine 226. Thenumber of frames in a given sequence may vary from location to location,and may be determined based on the AI engine's capabilities to producethe desired inference accuracy. For example, in one embodiment, a videocamera may capture video of the off-boarding area of a ski lift at 30frames per second and 1050×1500 resolution. The video capture andprocessing performed by the video camera 102 and the video processingmodule 224 would extract all of the frames in the video, crop each frameto as small an area as is possible while still covering the monitoredarea, resize the images to, for example, 299×299 resolution, and buildsequences consisting of four frames, each 0.5 seconds apart. Thesesequences then represent a 2-second long view of what's happening in thescene, and become the input to the AI engine 226. How these sequencesmay be used is detailed in the image-sequence curation, training of theAI engine 226, and operationalization of the system described herein.

In order to train the AI engine 226 to correctly determine if a sequenceof frames represents normal conditions or a developing problem situation(e.g., an abnormal on-boarding or off-boarding process) the AI engine226 is taught what is normal and what is abnormal. This is accomplishedby “training” the model used by the AI engine 226. In one embodiment,the AI model is trained via a process of supervised learning. Supervisedlearning is the machine learning task of inferring a function fromlabeled training data. For example, when using supervised learning, thetraining the AI engine 226 may involve showing the AI model used by theAI engine 226 a sequence of frames and telling the AI model “this isnormal” or “this is a problem”. In other words, the model is given manyexamples of “normal” and “not normal” sequences of frames, from which it“learns” what constitutes normal and what constitutes a developingproblem situation.

The process of creating these frame sequences and labeling them with thedesired outcome is called curation. For example, a sequence of fourframes where the skiers all ski away normally would be labeled as“normal”, or “green, or some other identifier to which the InferenceProcessing module 228 can react. Conversely, a sequence of framesshowing a skier falling in the off-boarding area and entirely blockingthe path for subsequent skiers represents an abnormal off-boardingprocess and this may be labeled as something like “red”, which theinference processing module 228 could interpret as “stop the lift”. Asequence of events that represents an abnormal on-boarding oroff-boarding process may include, but is not limited to, one or more of:a lift rider falling while on-boarding the ski lift; a lift riderfalling while off-boarding the ski lift; a lift rider lying or sittingthe ground in a lift chair loading zone; a lift rider being dragged by aski lift chair; a ski pole or other equipment of a lift rider beingcaught in a ski lift chair; a lift rider proceeding to on-board the skilift late; a lift rider failing to exit off-boarding area; a lift riderbeing in an abnormal position on a lift chair after on-boarding the skilift; a lift rider starting to slip off a lift chair after on-boardingthe ski lift; a lift rider leaving a lift chair loading zone as a liftchair is approaching the lift rider; a lift rider under a pre-determinedsize in a lift chair loading zone as a lift chair is approaching thelift rider; a number of lift riders being in a lift chair loading zonethat is over a pre-determined limit of lift riders for a lift chair; alift rider facing a wrong direction in a lift chair loading zone as thelift chair is approaching the lift rider; a lift rider being in anincorrect position within a lift chair loading zone as the lift chair isapproaching the lift rider; an incorrect number of lift ridersoff-boarding a lift chair compared to a number of lift riders thaton-boarded the lift chair; an incorrect number of lift riders on a liftchair compared to a number of lift riders that were in a lift chairloading zone for the lift chair when the lift chair arrived foron-boarding; a mechanical problem with a lift chair of the ski lift; amechanical problem with operation of the ski lift; and a problem with astructural element of the ski lift.

Frequently, curation is a manual process, though in some embodiments, itis automated. For example, existing video footage of people on- andoff-boarding from ski lifts may be processed by the video processingmodule 224 into frame sequences and manually curated to build the set ofsequences on which the model is trained. When training the AI engine226, many thousands of curated sequences will be used, so that the modelsees thousands of scenarios, increasing its accuracy.

In some embodiments, the AI engine may be trained using active machinelearning in which a learning algorithm interactively queries the user(or some other information source) to obtain the desired outputs at newdata points. In statistics literature this is sometimes also calledoptimal experimental design. The learning algorithm of the AI engine 226may actively query the user/teacher for labels. This is a type ofiterative supervised learning. Since the learner chooses the examples,the number of examples to learn a concept can often be much lower thanthe number required in normal supervised learning. Other embodiments mayuse multi-label active learning, hybrid active learning and activelearning in a single-pass context. Some embodiments combine conceptsfrom the field of machine learning (e.g., conflict and ignorance) withadaptive, incremental learning policies in the field of online machinelearning.

In some embodiments, the AI engine 226 may be trained using unsupervisedlearning, or may perform supervised or unsupervised learning duringoperation. Unsupervised learning is a type of machine learning algorithmused to draw inferences from datasets consisting of input data withoutlabeled responses. In some embodiments, the AI engine 226 may useunsupervised learning in that the data set used for training may beimages or sequences of images that are not labeled or curated. Forexample, when performing unsupervised learning, the ski lift problemdetection system 104 may use cluster analysis, which is used forexploratory data analysis to find hidden patterns or grouping in the keyentities and relationships extracted from the images, sequences ofimages or other data used to train the AI engine 226.

The AI engine 226 is able to “see” what is happening in a given videoframe and examine a sequence of frames such that a given frame is seenin context of the previous frames. This is important in that a givensituation, e.g., a single skier skiing away from the lift, can be goodor bad depending on what was happening before the skier skied away. If,for example, three people got off the lift but only one skied away,there was possibly a problem wherein two skiers fell down (e.g., “red”scenario). Alternatively, if only one skier got off the lift and thatone skier skied away, then all is good (e.g., “green” scenario). Thiscontext-based awareness can be accomplished, for example, by adding aLong Short-Term Memory (LSTM) component to the AI model used by the AIengine 226. LSTMs are a kind of recurrent artificial neural networkcapable of learning long-term dependencies. LSTMs also have a chain likestructure, but the repeating module has a different structure than morebasic recurrent neural network blocks. In particular, the LSTM recurrentartificial neural network blocks include a “cell state”. The cell statecan be analogized to a conveyor belt that runs straight down the entirechain of blocks, with only some minor linear interactions. This allowsinformation to easily flow along it unchanged. Instead of having asingle neural network layer, there are multiple layers interacting in aspecific way to enable the long-term dependencies.

In particular, there is a sigmoid layer called the “forget gate layer”which decides what information gets discarded from the cell state. Thismay also be referred to as a “forgetting artificial neural network”that, in one embodiment, determines which of the previously generateddeterminations of the AI engine 226 regarding what is a “red” scenarioand what is a “green” scenario are reintroduced by the LSTM in thesubsequent iterations of determinations.

The LSTM also includes a tan h layer that creates a vector of newcandidate values that could be added to the cell state. This may also bereferred to as a “generation artificial neural network” that, in oneembodiment, generates determinations during a number of iterations ofthe LSTM of the AI engine 226.

The LSTM also decides what new information is going to be stored in thecell state. This is performed by a sigmoid layer called the “input gatelayer” that decides which values will be updated. This may also bereferred to as an “ignoring artificial neural network” that, in oneembodiment, filters the generated predictions of the LSTM of the AIengine 226 based on relevancy. The memory structure reintroduces (adds)these previously generated predictions from this layer to be used insubsequent iterations of predictions.

There also exists another sigmoid gate which decides what parts of thecell state are going to be output. The cell state is then output to atan h layer (to push the values to be between −1 and 1) and thenmultiplied by the output of the sigmoid gate, so that only the partsdecided by the previous layers are output from the block. This may alsobe referred to as a “selection artificial neural network” that, in oneembodiment, determines which of the generated predictions of the LSTM torelease to an output system and which of the predictions to retaininternally in the artificial neural networks only for use in thesubsequent iterations of predictions. Such an LSTM artificial neuralnetwork architecture is one example embodiment and other LSTMarchitectures that enable learning of learning long-term dependenciesmay also be used by the AI engine 226.

In one example embodiment example, “seeing” what is happening in a givenvideo frame can be accomplished by building a Convolutional NeuralNetwork (CNN) component using the Tensorflow framework. The CNN maycomprise of convolutional layers, pooling layers, and fully-connectedlayers. The number of nodes in each of these layers can be adjusted totune the “seeing” part of the model to improve performance.Alternatively, other constructs, such as a Capsule Network built withthe Tensorflow or Pytorch framework could be used.

The Tensorflow or Pytorch frameworks may also be used to build the LSTMlayer of the model, using constructs available in those frameworks. Inthis layer, one can adjust the number of frames in the examined sequenceas well as the number of nodes in the LSTM layer to tune the AI modelused by the AI engine 226. The tuning adjustments described herein,i.e., the number of nodes, or number of frames, etc., are calledhyperparameters. Hyperparameter tuning is an important aspect to AIengine 226 development and training.

The AI engine 226 in the system may be trained to identify variousscenarios using curated data. Such scenarios may include “normal”,“problem might be developing—keep an eye on this”, “minor problem”, and“major problem”. Other types and categories and labels of scenarios mayalso or instead be used in various different embodiments. Training theAI model used by the AI engine 226 may be accomplished by feeding the AImodel thousands of curated data sets and letting the model, via amultitude of computations, determine what it thinks is happening in eachdata set. The model's answer is then compared to the correct answer forthe data set. For example, the model may predict that a given data setis a “minor problem” condition, but it's really a “normal” condition. Ifthe model's answer is incorrect, adjustments are made to the model via aprocess called backpropagation, and the process is repeated. Thissequence of compute-compare-adjust is repeated many times until themodel produces the desired prediction accuracy.

In some embodiments, the data set used for training is a large number ofimages grouped into the individual categories of “normal”, “problemmight be developing—keep an eye on this”, “minor problem”, and “majorproblem”. In addition to the training data set, a separate group ofimages with the same categories may be kept and is called the validationset. This data set is not used during training, but for testing thequality of the trained model. In some embodiments, data augmentation isapplied to the data set such has random crop (rotation & shifting),resolution scaling, grayscale conversion, color distortion, vertical andhorizontal flip, shearing, and stretching. The validation set is used atevery “epoch” of training. One epoch is a complete pass through all thetraining data. Training involves many epochs as the AI engine 226 passesthrough the data set, adjusting weights to make the classifier moreaccurate. After one epoch, the validation data set is used and thesystem can get an accuracy percentage from it. This is important becausethe system can train so much that the neural network starts memorizingpixels and irrelevant information from the data set, called overfitting.But then because the system just started memorizing color or pixels, itwill underperform on data the system has not seen before, the validationset. So to prevent the neural network from overfitting, the system mayuse a validation set as a threshold of when to stop training.

For example, in one embodiment, there may be a separate folder of images(e.g., stored in the other data storage 220) that the neural network ofthe AI engine 226 has not seen before. The neural network is pointed tothat folder and the neural network will initialize the training process.For training, one embodiment uses Nvidia Digits because it has agraphical user interface that can adjust learning rates and trainingsets.

In one embodiment, the machine learning model that receives the highestvalidation accuracy is sent to AI engine 226 of the ski lift problemdetection system 104 periodically, such as every day. The data may betrained on one or more remote servers due to alleviate large hardwarerequirements. For example, such processing may be hosted by Amazon WebServices (AWS). In other embodiments, this training may be done locallyby the AI engine 226. New data can be uploaded for training every dayand a new model is sent to the AI engine 226 after meeting accuracyrequirements. In this way, the AI engine 226 can fix mistakes in itsaccuracy and increase confidence on correct classifications. Forexample, the new data may be the images of operations of the ski lifton-boarding and off-boarding areas. Since the ski lift problem detectionsystem 104 initially has never seen the ski lift and the environment theski lift is in, it can have low accuracy. As more images of the ski lifton-boarding and off-boarding areas are uploaded for training, accuracyincreases. A module of the ski lift problem detection system operationmanager 222 (e.g., the inference processing module 228) or otherprograms 230 may cause the ski lift problem detection system 104 toautomatically upload these images, at the discretion of the ski lift orski area operator. For example, the inference processing module 228 mayfeed the image sequences, along with their inferences to a module in 222that would do the uploading.

The more processing capability the AI engine 226 has, the morefrequently model updates can be done for that AI engine 226. Note thatin some embodiments, the classifications or categorization (also knownas “inference” in deep learning”) occur at a remote server incommunication with the ski lift problem detection system 104 overcommunication system 108, and not on the ski lift problem detectionsystem 104. Because a remote server may have hardware with higherprocessing power than the deployed ski lift problem detection system104, training flows may involve the ski lift problem detection system104 uploading a new data set, training the data set on a bigger,high-accuracy convolutional neural network that automaticallycategorizes the data, and then training a smaller neural network on thenewly categorized data that can be sent to the deployed ski lift problemdetection system 104 from the server. The new data set may be used totrain a larger convolutional neural network. But the point of the large,high-accuracy convolutional neural network is to automaticallycategorize the new data set and fix the mistakes of the small neuralnetwork. The categorized data set is then used to train the small neuralnetwork, and a new version is then deployed.

Training systems and methods for improved operations of ski lifts mayinvolve a neural network trained on a database of images, then lockingthe top layers, removing the last connected layer, creating a new lastlayer with the desired number of categories, and adjusting thehyperparameters. Many neural networks are publicly available online. Alist of them for the Caffe framework is athttps://github.com/BVLC/caffe/wiki/Model-Zoo. One can download them fromgithub directly. In one example, SqueezeNet may be used, which is madeto run on mobile devices and does not require a large amount of power,but gives up some accuracy. However, different neural networks areavailable and contemplated for performing the functionality describedherein.

A neural network for images typically includes of convolutional layersconnected to each other. Locking the top layers means those layers arenot allowed to learn or change their weights. It is found that aftertraining large diverse data sets, the top layers do not change a largeamount. This is because the top layers are those that start recognizingbasic shapes and lines in pictures, which is useful for many differentimage recognition tasks. Locking the top layers does not allow theweights to change on the top layers, but allows the bottom layer tolearn and change weights. Since the bottom layer can learn and changeits weights, it can be made specific to one's image recognition task.

Adjusting hyperparameters refers to higher level “knobs” that can bechanged and adjusted for training. One common hyperparameter is thelearning rate. By making the learning rate large, the neural network canlearn quickly. By making the learning rate small, the neural networkwill learn slower and can get stuck in a certain local minima, resultingin a fixed, substandard accuracy. In particular, the problem is that alow learning rate causes the AI system to get stuck in a local low spotin the loss curve. An example will be provided to explain this. When thesystem starts training, a larger learning rate can be set. The accuracycan jump from 0% to 60%, then down to 30%, then up to 70%. It can varywidely. However, one can take the model that hit 70% and set a smallerlearning rate. Then the neural network can creep up from 70% to 75%, to78% accuracy, etc.

Different frameworks for neural networks involve different amounts ofcoding. Some frameworks allow one to describe the network in python, soone can connect the convolutional layers together with python. Caffeuses protobuf, which is what may be used to adjust the layers and changelearning rates. It is basically a structured way of describing anetwork. There currently exist many machine learning tools which arereferred to as “frameworks”. They are middleware which make it easier tobuild a neural network and train. Tensorflow is one example, but Caffemay be preferable in some embodiments because it handles images well.

Although in some embodiments, the ski lift problem detection system 104implements an algorithm to remove duplicate images, the ski lift problemdetection system 104 may save transitions for training because it addsmore diversity to the data set. For example, when a classificationchanges from “minor problem” to “major problem”, the ski lift problemdetection system 104 will save both images or sequences for training.Movement detection can be implemented by watching transition changes inclassifications. Also, detecting if the image is a duplicate of theprevious image is a straightforward way of detecting movement. Althoughin one embodiment the data set only includes images, in otherembodiments, models can also be trained on video and sound. Detectingthe difference between the sound of a lift rider successfullyon-boarding and a lift rider falling may be useful for aidingclassification of problem scenarios.

In one embodiment, the ski lift problem detection system 104 may be adevice or electronic equipment that is operable to receive and processimages and/or video of the ski lift and communicate data regarding theimages and/or video as described herein. Note that one or more generalpurpose or special purpose computing systems/devices may be used tooperate the ski lift problem detection system 104, store informationregarding the ski lift problem detection system 104, store adjustmentsettings and communicate with the video camera 102, the ski-lift motorcontroller 106 and/or the ski lift alert system 110. Such adjustmentsettings may be stored in the ski lift problem detection system problemdetection rules and settings storage 216. In one embodiment, thecomputing system may be a local system-on-a-chip (SoC) system, such as,for example, Raspberry Pi. However, other processors and computingplatforms may be used. In addition, the ski lift problem detectionsystem 104 may comprise one or more distinct computing systems/devicesand may span distributed locations. Furthermore, each block shown mayrepresent one or more such blocks as appropriate to a specificembodiment or may be combined with other blocks. Also, the ski liftproblem detection system operation manager 222 may be implemented insoftware, hardware, firmware, or in some combination to achieve thecapabilities described herein.

In the embodiment shown, ski lift problem detection system 104 comprisesa computer memory (“memory”) 201, a display 202 (including, but notlimited to a light emitting diode (LED) panel, liquid crystal display(LCD), touch screen display, etc.), one or more Central Processing Units(“CPU”) 203, Input/Output devices 204 (e.g., button panel, keyboard,mouse, RF or infrared receiver, universal serial bus (USB) ports, othercommunication ports, and the like), other computer-readable media 205,and network connections 206. The ski lift problem detection systemoperation manager 222 is shown residing in memory 201. In otherembodiments, some portion of the contents and some, or all, of thecomponents of the ski lift problem detection system operation manager222 may be stored on and/or transmitted over the other computer-readablemedia 205. The components of the ski lift problem detection system 104and ski lift problem detection system operation manager 222 preferablyexecute on one or more CPUs 203 and perform or otherwise facilitatecapturing digital video of one or more of on-boarding and off-boardingoperations of a ski lift; generating a plurality of digital images ofthe one or more of on-boarding and off-boarding operations of the skilift based on the captured digital video; automatically detecting, inreal-time as the digital video is being captured, as the ski lift isoperating, a potential problem situation in one or more of anon-boarding area and an off-boarding area of the ski lift based on theplurality of digital images; and initiating an action, by the at leastone computer processor, as the ski lift is operating, to address thepotential problem situation in the one or more of the on-boarding areaand off-boarding area of the ski lift while the potential problemsituation still exists, as described herein. In other embodiments, otherprocessing devices and configurations may be used, including, but notlimited to, graphics processing units (GPU), ASICs and embedded CPU/GPUblocks, Neural Processing Units (NPU), Intelligence Processing Units(IPU), and Deep Learning Accelerators (DLA). Such processors often havefunctionality that accelerates the execution of matrix math forconvolutional neural networks enabling the systems and methods forimproved operations of ski lifts described herein to classify and/ortrain faster. The ski lift problem detection system operation manager222 may operate as, be part of, or work in conjunction and/orcooperation with various software applications stored in memory 201. Theski lift problem detection system operation manager 222 also facilitatescommunication the video camera 102, the ski-lift motor controller 106,the ski lift alert system 110, peripheral devices and/or other systems,such as a remote server, via the I/O devices 204 and/or the networkconnections 206.

Recorded or buffered digital video of the ski lift areas received mayreside on the other data repository 220, for storing, processing,analyzing, communicating and displaying of the received images fromdigital video captured by the camera 232. The other data repository 220may also store various video and image metadata associated with therecorded or buffered video and images in the other data repository 220,such as that including, but not limited to, resolution indicators,format indicators, tags, codes, labels, curation information,identifiers, format indicators, timestamps, user identifications,authorization codes, digital signatures, etc.

The video processing module 224 is configured to decompose the receivedvideo into individual frames, crop and resized the frames and thenassemble the frames in manageable sequences to be fed to the AI engine226. The AI engine 226 is configured to analyze the plurality of digitalimages (e.g., sequences of images) and detect in real-time as thedigital video is being captured, and as the ski lift is operating, apotential problem situation in one or more of an on-boarding area and anoff-boarding area of the ski lift based on the plurality of digitalimages. This may include, for example, determining which of theplurality of digital images of the ski lift area generated correspondsto which of a plurality of different recognized classifications or typesof potential problems. The AI engine 226 may be trained to recognizesuch images or sequence of images by training, for example, aconvolutional multi-layer neural network either resident within thememory 201, other programs 230 and/or on a remote server, to detect thepotential problem situation. This may be accomplished by training theconvolutional multi-layer neural network using a database of images ordifferent sequences of images representative of corresponding differentpotential problem situations, wherein each image is tagged or labeled asassociated with a particular potential problem situation. For example,each different potential problem situation may be designated asrepresentative of a corresponding different one the states: “normal”,“problem might be developing —keep an eye on this”, “minor problem”, and“major problem”.

The video processing module 224 may also decode, decompress, format,translate, perform digital signal processing, adjust data rate and/orcomplexity or perform other processing on the data representing receivedvideo and/or images of the ski lift area as applicable for processingand, in some embodiments, presenting the data in real time or near realtime as it is being received by the ski lift problem detection system104. Such video sequences detected as representing a potential problemsituations and corresponding alerts may also be communicated, viacommunication system 108, to remote devices, such as mobile devices,smartphones or tablets in real-time or near real-time as the potentialproblem situation is occurring.

Other code or programs 230 (e.g., further audio/video processingmodules, a program guide manager module, a Web server, and the like),and potentially other data repositories, such as data repository 220 forstoring other data (user profiles, preferences and other configurationdata, etc.), also reside in the memory 201, and preferably execute onone or more CPUs 203. Of note, one or more of the components in FIG. 2may or may not be present in any specific implementation. For example,some embodiments may not provide other computer readable media 205 or adisplay 202.

In some embodiments, the ski lift problem detection system 104 and skilift problem detection system operation manager 222 includes anapplication program interface (“API”) that provides programmatic accessto one or more functions of the ski lift problem detection system 104and operation manager 222. For example, such an API may provide aprogrammatic interface to one or more functions of the ski lift problemdetection system operation manager 222 that may be invoked by one of theother programs 230, the video camera 102, the ski lift alert system 10,the ski lift motor controller 106, a mobile device (not shown), a server(not shown) or some other module or remote system. In this manner, theAPI may facilitate the development of third-party software, such asvarious different service applications, user interfaces, plug-ins,adapters (e.g., for integrating functions of the ski lift problemdetection system operation manager 222 into desktop or mobileapplications), and the like to facilitate systems and methods forimproved operations of ski lifts using the ski lift problem detectionsystem 104.

In an example embodiment, components/modules of the ski lift problemdetection system 104 and the ski lift problem detection system operationmanager 222 are implemented using standard programming techniques. Forexample, the ski lift problem detection system operation manager 222 maybe implemented as a “native” executable running on the CPU 203, alongwith one or more static or dynamic libraries. In other embodiments, theski lift problem detection system 104 and ski lift problem detectionsystem operation manager 222 may be implemented as instructionsprocessed by a virtual machine that executes as one of the otherprograms 230. In general, a range of programming languages known in theart may be employed for implementing such example embodiments, includingrepresentative implementations of various programming languageparadigms, including but not limited to, object-oriented (e.g., Java,C++, C#, Visual Basic.NET, Smalltalk, and the like), functional (e.g.,ML, Lisp, Scheme, and the like), procedural (e.g., C, Pascal, Ada,Modula, and the like), scripting (e.g., Perl, Ruby, Scratch, Python,JavaScript, VBScript, and the like), or declarative (e.g., SQL, Prolog,and the like).

In a software or firmware implementation, instructions stored in amemory configure, when executed, one or more processors of the ski liftproblem detection system 104 to perform the functions of the ski liftproblem detection system operation manager 222. The instructions causethe CPU 203 or some other processor, such as an I/Ocontroller/processor, to perform the processes described herein.

The embodiments described above may also use well-known or othersynchronous or asynchronous client-server computing techniques. However,the various components may be implemented using more monolithicprogramming techniques as well, for example, as an executable running ona single CPU computer system, or alternatively decomposed using avariety of structuring techniques known in the art, including but notlimited to, multiprogramming, multithreading, client-server, orpeer-to-peer (e.g., Bluetooth® wireless technology providing acommunication channel between the ski lift problem detection system 104and a mobile device), running on one or more computer systems eachhaving one or more CPUs or other processors. Some embodiments mayexecute concurrently and asynchronously, and communicate using messagepassing techniques. Equivalent synchronous embodiments are alsosupported by a ski lift problem detection system operation manager 222implementation. Also, other functions could be implemented and/orperformed by each component/module, and in different orders, and bydifferent components/modules, yet still achieve the functions of the skilift problem detection system 104 and operation manager 222.

In addition, programming interfaces to the data stored as part of theski lift problem detection system 104 and ski lift problem detectionsystem operation manager 222, can be available by standard mechanismssuch as through C, C++, C#, and Java APIs; libraries for accessingfiles, databases, or other data repositories; scripting languages; orWeb servers, FTP servers, or other types of servers providing access tostored data and machine learning models. The ski lift problem detectionsystem problem detection rules and settings storage 216 and other data220 may be implemented as one or more database systems, file systems, orany other technique for storing such information, or any combination ofthe above, including implementations using distributed computingtechniques.

Different configurations and locations of programs and data arecontemplated for use with techniques described herein. A variety ofdistributed computing techniques are appropriate for implementing thecomponents of the illustrated embodiments in a distributed mannerincluding but not limited to TCP/IP sockets, RPC, RMI, HTTP, and WebServices (XML-RPC, JAX-RPC, SOAP, and the like). Other variations arepossible. Other functionality could also be provided by eachcomponent/module, or existing functionality could be distributed amongstthe components/modules in different ways, yet still achieve thefunctions of the ski lift problem detection system operation manager222.

Furthermore, in some embodiments, some or all of the components of theski lift problem detection system 104 and operation manager 222 may beimplemented or provided in other manners, such as at least partially infirmware and/or hardware, including, but not limited to one or moreapplication-specific integrated circuits (“ASICs”), standard integratedcircuits, controllers (e.g., by executing appropriate instructions, andincluding microcontrollers and/or embedded controllers),field-programmable gate arrays (“FPGAs”), complex programmable logicdevices (“CPLDs”), and the like. Some or all of the system componentsand/or data structures may also be stored as contents (e.g., asexecutable or other machine-readable software instructions or structureddata) on a computer-readable medium (e.g., as a hard disk; a memory; acomputer network, cellular wireless network or other data transmissionmedium; or a portable media article to be read by an appropriate driveor via an appropriate connection, such as a flash memory device) so asto enable or configure the computer-readable medium and/or one or moreassociated computing systems or devices to execute or otherwise use, orprovide the contents to perform, at least some of the describedtechniques.

FIG. 3 is a diagram of an example ski lift at an off-boarding area 302showing a ski lift attendant 304 in a first position. As shown in FIG. 3, the lift attendant 304 is out of position to quickly slow or stop thelift if necessary in case a problem situation arises when the liftriders are off-boarding the ski lift. Advantageously, the ski liftproblem detection system 104 may automatically slow or stop the ski liftupon detection by the ski lift problem detection system 104 of apotential problem situation represented in captured video sent from thevideo camera 102 to the ski lift problem detection system 104. The sceneshown in FIG. 3 is from an example viewpoint of the camera 102 and theimage shown in FIG. 3 may be a frame of a sequence of video frames ofthe captured video sent from the video camera 102.

FIG. 4 is a diagram of an example ski lift at an off-boarding area 402showing a ski lift attendant 404 in a second position. As shown in FIG.4 , the lift attendant 404 has their back to the action and would notsee if the off-boarding skier had fallen when off-boarding the ski liftchair and thus is would not be able to recognize the problem situationin a timely manner and quickly slow or stop the lift. Advantageously,the ski lift problem detection system 104 would automatically slow orstop the ski lift if the skier had fallen when off-boarding the ski liftchair upon detection by the ski lift problem detection system 104 basedon captured video sent from the video camera 102 to the ski lift problemdetection system 104. The scene shown in FIG. 3 is from an exampleviewpoint of the camera 102 and the image shown in FIG. 3 may also be aframe of a sequence of video frames of the captured video sent from thevideo camera 102.

FIG. 5 is a block diagram illustrating a flow of operations betweenvarious elements of an example system for improved operations of skilifts, according to one example embodiment.

The video camera 102 monitors one or more lift riders 500 in anon-boarding and/or off-boarding area of the ski lift. The videoprocessing module 224 of the ski lift problem detection system 104extracts individual image frames from video the feed and then may crop,resize, and assemble them into a sequence for consumption by the AIengine 226. Some of these data may be persisted for use in furthertraining and refining the AI engine 226 as described herein.

The AI engine 226 receives sequences of images and infers the situationin the scene. Such inferences may be based on the training of the AImodel and may be specific to a given lift. For example, parametersassociated with a given lift may be stored by the ski lift problemdetection system 104 in the problem detection and rules settings storage216 and accessible from the problem detection and rules settings storage216. In some embodiments, the problem detection and rules settingsstorage 216 may be located remotely from the ski lift problem detectionsystem 104 and accessible via communication system 108. The problemdetection and rules settings storage 216 may store rules, settings andparameters for a plurality of different ski lifts and may be accessibleby a plurality of corresponding ski lift problem detection systems ofdifferent ski lifts.

The inference processing module 228 receives the inference value fromthe AI engine 226 and takes or initiates an appropriate action, if any,as the ski lift is operating, to address the potential problemsituation. In various different example embodiments, such actions canrange from alerting of the human attendant to interacting with the liftmotor controller to slowing or stopping the ski lift.

The ski lift alert system 110 may be a sound-based, light-based, ormessage-based alerting system to prompt the attendant to be ready totake action if necessary. For example, the inference processing module228 may send an audio alert, a message-based alert and/or a light-basedalert to one or more of: a device located on the ski lift, an electronicmessage board, an alarm on or near the ski lift, a bell on or near theski lift, a siren on or near the ski lift, an electronic sign or displayon or near the ski lift, a mobile device of the attendant, a remotesystem of the ski lift operator, and a remote system or device of safetypersonnel or ski patrol.

The inference processing module 228 interacts with the ski lift motorcontroller 106 to slow or stop the lift depending on the operationalrules for the lift. For example, such operational rules may be stored inand accessible from the problem detection rules and settings storage216, which, in some embodiments, may be located remotely from the skilift problem detection system 104 and accessible via communicationsystem 108. The motor controller may be an electrical, mechanical orelectro-mechanical device that is operable to receive electrical signalsto control one or more motors and/or braking systems of the ski lift.The operation of a ski lift may include a looped cable that spansbetween two large pulleys at each end (the bull wheels). Towers inbetween support the chairs (the carriers) as they travel up themountain. At the towers the cable runs through sheaves attached below orabove the towers depending on various conditions. Modern ski lifts relyon electric motors to turn the bull wheels. Most also have secondarybackup diesel power motors also, for safety. The power and motor may belocated at the top or bottom of the chair lift depending on engineeringrequirements. Power and motor control utilizes electrical contacts andthe ski lift motor controller 106 may include electrical, mechanical orelectro-mechanical devices to control electrical connectivity of theelectrical contacts and circuits to control lift drive systems,starting, stopping, operation, speed and direction of the motor and alsobraking systems. The particular signals and commands that the inferenceprocessing module 228 may use to control different motor controllers anddrive systems of various different ski lift motors and systems may bestored in and accessible from the problem detection rules and settingsstorage 216.

FIG. 6 is a flow diagram of a computerized method 600 for improved skilift operations, according to one example embodiment.

At 602, the system captures digital video of one or more of on-boardingand off-boarding operations of a ski lift.

At 604, the system generates, as the ski lift is operating, a pluralityof digital images of the one or more of on-boarding and off-boardingoperations of the ski lift based on the captured digital video.

At 606, the system automatically detects, in real-time as the digitalvideo is being captured and as the ski lift is operating, a potentialproblem situation in one or more of an on-boarding area and anoff-boarding area of the ski lift based on the plurality of digitalimages.

At 608, the system initiates an action, as the ski lift is operating, toaddress the potential problem situation in the one or more of theon-boarding area and off-boarding area of the ski lift while thepotential problem situation still exists.

FIG. 7 is a flow diagram of a computerized method 700 for initiating anaction to address a potential problem situation in one or more of anon-boarding area and off-boarding area of a ski lift useful in improvedski lift operations, according to one example embodiment.

At 702, the system compares a detected potential problem situation to aplurality of potential problem situations to determine a match of thedetected potential problem situation to a matching one of the pluralityof potential problem situations.

At 704, the system selects an action to initiate according to a ruleassociated with the matching one of the plurality of potential problemsituations.

At 706, the system initiates the selected action to address thepotential problem situation in the one or more of the on-boarding andoff-boarding area of the ski lift while the potential problem situationstill exists.

FIG. 8 is a flow diagram of a computerized method 800 for improved skilift operations, according to another example embodiment.

At 802, the system automatically detects, as a ski lift is operating, apotential problem situation in one or more of an on-boarding andoff-boarding area of the ski lift based on a plurality of digital imagesfrom captured digital video of operation of the ski lift. A sequence ofevents that represents an abnormal on-boarding or off-boarding processthat may be indicative of potential problem situation includes, but isnot limited to, one or more of: a lift rider falling while on-boardingthe ski lift; a lift rider falling while off-boarding the ski lift; alift rider lying or sitting the ground in a lift chair loading zone; alift rider being dragged by a ski lift chair; a ski pole or otherequipment of a lift rider being caught in a ski lift chair; a lift riderproceeding to on-board the ski lift late; a lift rider failing to exitoff-boarding area; a lift rider being in an abnormal position on a liftchair after on-boarding the ski lift; a lift rider starting to slip offa lift chair after on-boarding the ski lift; a lift rider leaving a liftchair loading zone as a lift chair is approaching the lift rider; a liftrider under a pre-determined size in a lift chair loading zone as a liftchair is approaching the lift rider; a number of lift riders being in alift chair loading zone that is over a pre-determined limit of liftriders for a lift chair; a lift rider facing a wrong direction in a liftchair loading zone as the lift chair is approaching the lift rider; alift rider being in an incorrect position within a lift chair loadingzone as the lift chair is approaching the lift rider; an incorrectnumber of lift riders off-boarding a lift chair compared to a number oflift riders that on-boarded the lift chair; an incorrect number of liftriders on a lift chair compared to a number of lift riders that were ina lift chair loading zone for the lift chair when the lift chair arrivedfor on-boarding; a mechanical problem with a lift chair of the ski lift;a mechanical problem with operation of the ski lift; and a problem witha structural element of the ski lift.

At 804, the system initiates an action, as the ski lift is operating, toaddress the potential problem situation in the one or more of theon-boarding and off-boarding area of the ski lift while the potentialproblem situation still exists.

While various embodiments have been described herein above, it is to beappreciated that various changes in form and detail may be made withoutdeparting from the spirit and scope of the invention(s) presently orhereafter claimed.

The invention claimed is:
 1. A computerized method for improved ski liftoperations, comprising: capturing, by at least one computer processor,digital video of one or more of on-boarding and off-boarding operationsof a ski lift; generating, by at least one computer processor, as theski lift is operating, a plurality of digital images of the one or moreof on-boarding and off-boarding operations of the ski lift based on thecaptured digital video, wherein the plurality of digital images includessequences of video frames, wherein each sequence of the sequences ofvideo frames includes a plurality of individual video framesrepresenting a sequence of events; automatically detecting, by at leastone computer processor, in real-time as the digital video is beingcaptured, as the ski lift is operating, a potential problem situationrepresented by an abnormal position of a lift rider while on-boarding oroff-boarding the ski lift in one or more of an on-boarding area and anoff-boarding area of the ski lift based on the plurality of digitalimages, wherein the automatically detecting the potential problemsituation includes: analyzing the plurality of digital images as the skilift is operating, including recognizing in the plurality of digitalimages a sequence of events that represents the abnormal position of alift rider resulting in an abnormal on-boarding or off-boarding processat a scene of the ski lift that includes the one or more of anon-boarding area and an off-boarding area of the ski lift; anddetermining, based on a combined sequence of video frames of at leastone of the sequences of video frames, whether a sequence of eventsrepresented by the at least one of the sequences of video framesindicates a problem situation represented by an abnormal position of alift rider while on-boarding or off-boarding the ski lift, wherein thesequence of events that represents the abnormal position of a lift riderresulting in the abnormal on-boarding or off-boarding process includes alift rider facing a wrong direction in a lift chair loading zone as thelift chair is approaching the lift rider; and initiating an action, bythe at least one computer processor, as the ski lift is operating, toaddress the potential problem situation in the one or more of anon-boarding area and off-boarding area of the ski lift while thepotential problem situation still exists.
 2. The method of claim 1further comprising: analyzing, by at least one computer processor, theplurality of digital images as the ski lift is operating, wherein theautomatically detecting the potential problem situation is based on theanalysis of the plurality of digital images.
 3. The method of claim 1wherein the sequence of events that represents the abnormal position ofthe lift rider resulting in the abnormal on-boarding or off-boardingprocess includes a lift rider falling while on-boarding the ski lift. 4.The method of claim 1 wherein the analyzing the plurality of digitalimages as the ski lift is operating, includes: analyzing the pluralityof digital images using an artificial intelligence engine trained toassess sequences of digital images and identify normal positions of thelift rider and potential problem situations taking place in a scene ofthe ski lift represented by abnormal positions of the lift rider thatincludes the one or more of an on-boarding area and an off-boarding areaof the ski lift.
 5. The method of claim 4 further comprising: trainingthe artificial intelligence engine, using sequences of training imagesof one or more of on-boarding and off-boarding operations of one or moreski lifts, to assess the sequences of digital images and identify normalpositions of the lift rider and potential problem situations takingplace in the scene represented by abnormal positions of the lift rider.6. The method of claim 1 wherein initiating the action to address thepotential problem situation in the one or more of the on-boarding andoff-boarding area of the ski lift while the potential problem situationstill exists includes: comparing the detected potential problemsituation represented by an abnormal position of a lift rider whileon-boarding or off-boarding the ski lift to a plurality of potentialproblem situations to determine a match of the detected potentialproblem situation to a matching one of the plurality of potentialproblem situations; selecting an action to initiate according to a ruleassociated with the matching one of the plurality of potential problemsituations; and initiating the selected action to address the potentialproblem situation in the one or more of the on-boarding and off-boardingarea of the ski lift while the potential problem situation still exists.7. The method of claim 1 wherein the initiating the action to addressthe potential problem situation in the one or more of the on-boardingand off-boarding area of the ski lift while the potential problemsituation still exists includes initiating electronic sending of asignal to a motor controller of the ski lift to slow down or stopoperation of the ski lift to address the potential problem situation inthe one or more of the on-boarding and off-boarding area of the skilift.
 8. The method of claim 1 wherein the initiating the action toaddress the potential problem situation in the one or more of theon-boarding and off-boarding area of the ski lift while the potentialproblem situation still exists includes initiating of electronic sendingof an alert for a ski lift attendant.
 9. The method of claim 1 whereinthe sending the alert for the ski lift attendant includes one or moreof: sending an audio alert, sending a message-based alert and sending alight-based alert.
 10. A system for improved operations of ski lifts,comprising: at least one memory; and at least one processor coupled tothe at least one memory, wherein the at least one memory hascomputer-executable instructions stored thereon that, when executed,cause the at least one processor to: capture digital video of one ormore of on-boarding and off-boarding operations of a ski lift; generate,as the ski lift is operating, a plurality of digital images of the oneor more of on-boarding and off-boarding operations of the ski lift basedon the captured digital video; automatically detect, as the ski lift isoperating, a potential problem situation represented by an abnormalposition of a lift rider while on-boarding or off-boarding the ski liftin one or more of the on-boarding and off-boarding area of the ski liftbased on the plurality of digital images, wherein the automaticallydetecting the potential problem situation includes analyzing theplurality of digital images as the ski lift is operating, includingrecognizing in the plurality of digital images a sequence of events thatrepresents the abnormal position of a lift rider resulting in anabnormal on-boarding or off-boarding process at a scene of the ski liftthat includes the one or more of an on-boarding area and an off-boardingarea of the ski lift, wherein the sequence of events that represents theabnormal position of a lift rider resulting in the abnormal on-boardingor off-boarding process includes a lift rider facing a wrong directionin a lift chair loading zone as the lift chair is approaching the liftrider; and initiate an action, as the ski lift is operating, to addressthe potential problem situation in the one or more of the on-boardingand off-boarding area of the ski lift while the potential problemsituation still exists.
 11. A ski lift controller comprising: a ski liftmotor controller of a ski lift; at least one memory; at least one videocamera; and at least one processor coupled to the at least one memory,the at least one camera and the ski lift motor controller, wherein theat least one memory has computer-executable instructions stored thereonthat, when executed, cause the at least one processor to: capture, viathe at least one video camera, digital video of one or more ofon-boarding and off-boarding operations of the ski lift; generate, asthe ski lift is operating, a plurality of digital images of the one ormore of on-boarding and off-boarding operations of the ski lift based onthe captured digital video; automatically detect, as the ski lift isoperating, a potential problem situation in one or more of theon-boarding and off-boarding area of the ski lift represented by anabnormal position of a lift rider while on-boarding or off-boarding theski lift based on the plurality of digital images, wherein theautomatically detecting the potential problem situation includesanalyzing the plurality of digital images as the ski lift is operating,including recognizing in the plurality of digital images a sequence ofevents that represents the abnormal position of a lift rider resultingin an abnormal on-boarding or off-boarding process at a scene of the skilift that includes the one or more of an on-boarding area and anoff-boarding area of the ski lift, wherein the sequence of events thatrepresents the abnormal position of a lift rider resulting in theabnormal on-boarding or off-boarding process includes a lift riderfacing a wrong direction in a lift chair loading zone as the lift chairis approaching the lift rider; and initiate an action, as the ski liftis operating, to address the potential problem situation in the one ormore of the on-boarding and off-boarding area of the ski lift while thepotential problem situation still exists.
 12. A non-transitorycomputer-readable storage medium, having computer executableinstructions stored thereon that, when executed by the at least oneprocessor, cause the at least one processor to: automatically detect, asa ski lift is operating, a potential problem situation in one or more ofan on-boarding and off-boarding area of the ski lift represented by anabnormal position of a lift rider while on-boarding or off-boarding theski lift based on a plurality of digital images from captured digitalvideo of operation of the ski lift, wherein the automatically detectingthe potential problem situation includes analyzing the plurality ofdigital images as the ski lift is operating, including recognizing inthe plurality of digital images a sequence of events that represents theabnormal position of a lift rider resulting in an abnormal on-boardingor off-boarding process at a scene of the ski lift that includes the oneor more of an on-boarding area and an off-boarding area of the ski lift,wherein the sequence of events that represents the abnormal position ofa lift rider resulting in the abnormal on-boarding or off-boardingprocess includes a lift rider facing a wrong direction in a lift chairloading zone as the lift chair is approaching the lift rider; andinitiate an action, as the ski lift is operating, to address thepotential problem situation in the one or more of the on-boarding andoff-boarding area of the ski lift while the potential problem situationstill exists.
 13. The non-transitory computer-readable storage medium ofclaim 12, wherein the computer executable instructions, when executed,further cause the at least one processor to: capture digital video ofthe one or more of on-boarding and off-boarding operations of a skilift; and generate, as the ski lift is operating, the plurality ofdigital images of the one or more of on-boarding and off-boardingoperations of the ski lift based on the captured digital video.
 14. Themethod of claim 1 wherein the automatically detecting in real-time asthe digital video is being captured, as the ski lift is operating, thepotential problem situation represented by the abnormal position of thelift rider in one or more of the on-boarding area and an off-boardingarea of the ski lift based on the plurality of digital images includesautomatically detecting in real-time as the digital video is beingcaptured, as the ski lift is operating, the potential problem situationin the one or more of an on-boarding area and an off-boarding area ofthe ski lift before boarding of the ski lift by the lift rider.
 15. Themethod of claim 1 wherein the automatically detecting in real-time asthe digital video is being captured, as the ski lift is operating, thepotential problem situation represented by the abnormal position of thelift rider in the one or more of an on-boarding area and an off-boardingarea of the ski lift based on the plurality of digital images includesautomatically detecting in real-time as the digital video is beingcaptured, as the ski lift is operating, the potential problem situationin an off-boarding area of the ski lift.
 16. The method of claim 1wherein the sequence of events that represents the abnormal position ofa lift rider resulting in the abnormal on-boarding or off-boardingprocess includes a lift rider falling while off-boarding the ski lift.17. The method of claim 1 wherein the sequence of events that representsthe abnormal position of a lift rider resulting in the abnormalon-boarding or off-boarding process includes a lift rider lying orsitting on the ground in a lift chair loading zone.
 18. The method ofclaim 1 wherein the sequence of events that represents the abnormalposition of a lift rider resulting in the abnormal on-boarding oroff-boarding process includes a lift rider being dragged by a ski liftchair.
 19. The method of claim 1 wherein the sequence of events thatrepresents the abnormal position of a lift rider resulting in theabnormal on-boarding or off-boarding process includes a ski pole orother equipment of a lift rider being caught in or under a ski liftchair.
 20. The method of claim 1 wherein the sequence of events thatrepresents the abnormal position of a lift rider resulting in theabnormal on-boarding or off-boarding process includes a lift riderproceeding to on-board the ski lift late.
 21. The method of claim 1wherein the sequence of events that represents an abnormal position of alift rider resulting in the abnormal on-boarding or off-boarding processincludes a lift rider leaving a lift chair loading zone as a lift chairis approaching the lift rider.
 22. The method of claim 1 wherein thesequence of events that represents the abnormal position of a lift riderresulting in the abnormal on-boarding or off-boarding process includes alift rider being in an incorrect position within a lift chair loadingzone as the lift chair is approaching the lift rider.
 23. The method ofclaim 1 wherein the determining, based on a combined sequence of videoframes of at least one of the sequences of video frames, whether asequence of events represented by the at least one of the sequences ofvideo frames indicates a problem situation represented by an abnormalposition of a lift rider while on-boarding or off-boarding the ski liftincludes examining the combined sequence of video frames such that agiven frame of the combined sequence of video frames is seen, by the atleast one processor, in context of the previous frames of the combinedsequence of video frames.
 24. The method of claim 1 wherein thedetermining, based on a combined sequence of video frames of at leastone of the sequences of video frames, whether a sequence of eventsrepresented by the at least one of the sequences of video framesindicates a problem situation represented by an abnormal position of alift rider while on-boarding or off-boarding the ski lift includesclassifying the combined sequence of video frames as representing aproblem situation, and further comprising: determining, by the at leastone processor, that the at least one of the sequences of video framesindicates a problem situation based on the classification of thecombined sequence of video frames as representing the problem situation.25. The method of claim 1 wherein the determining, based on a combinedsequence of video frames of at least one of the sequences of videoframes, whether a sequence of events represented by the at least one ofthe sequences of video frames indicates a problem situation representedby an abnormal position of a lift rider while on-boarding oroff-boarding the ski lift includes: before generating the plurality ofdigital images of the one or more of on-boarding and off-boardingoperations: training an AI model with normal on-boarding or off-boardingtraining sequences of video frames, wherein a combination of videoframes comprising each normal on-boarding or off-boarding trainingsequence represents a corresponding normal position of a lift riderwhile on-boarding or off-boarding the ski lift by indicating to the AImodel that, for each normal on-boarding or off-boarding trainingsequence of video frames, the combination of video frames comprising thenormal on-boarding or off-boarding training sequence represents a normalposition of a lift rider while on-boarding or off-boarding the ski lift;and training the AI model with abnormal on-boarding or off-boardingtraining sequences of video frames, wherein a combination of videoframes comprising each abnormal on-boarding or off-boarding trainingsequence represents a corresponding abnormal position of a lift riderwhile on-boarding or off-boarding the ski lift by indicating to the AImodel that, for each abnormal on-boarding or off-boarding trainingsequence of video frames, the combination of video frames comprising theabnormal on-boarding or off-boarding training sequence; and representsan abnormal position of a lift rider while on-boarding or off-boardingthe ski lift; and determining, using the AI model trained with thenormal on-boarding or off-boarding training sequences of video framesand with the abnormal on-boarding or off-boarding training sequences ofvideo frames, based on the combined sequence of video frames of the atleast one of the sequences of video frames included in the generatedplurality of digital images of the one or more of on-boarding andoff-boarding operations, whether the at least one of the sequences ofvideo frames included in the generated plurality of digital images ofthe one or more of on-boarding and off-boarding operations represents anevent that indicates a problem situation represented by an abnormalposition of a lift rider while on-boarding or off-boarding the ski lift.26. The method of claim 25 wherein the determining whether the at leastone of the sequences of video frames included in the generated pluralityof digital images of the one or more of on-boarding and off-boardingoperations represents an event that indicates a problem situationrepresented by an abnormal position of a lift rider while on-boarding oroff-boarding the ski lift includes detection of movement of the liftrider represented by the at least one of the sequences of video frameswhile on-boarding or off-boarding the ski lift.
 27. The method of claim1 wherein the potential problem situation is represented by an abnormalposition of a plurality of lift riders while on-boarding or off-boardingthe ski lift based on detected states of the plurality of lift riders.28. The method of claim 25 wherein the combination of video framescomprising each normal on-boarding or off-boarding training sequencerepresents a corresponding normal position of a plurality of lift riderswhile on-boarding or off-boarding the ski lift based on detected statesof the plurality of lift riders.